204 research outputs found
Disrupted orbital order and the pseudo-gap in layered 1T-TaS
We present a state-of-the-art density functional theory (DFT) study which
models crucial features of the partially disordered orbital order stacking in
the prototypical layered transition metal dichalcogenide 1T-TaS2 . Our results
not only show that DFT models with realistic assumptions about the orbital
order perpendicular to the layers yield band structures which agree remarkably
well with experiments. They also demonstrate that DFT correctly predicts the
formation of an excitation pseudo-gap which is commonly attributed to
Mott-Hubbard type electron-electron correlations. These results highlight the
importance of interlayer interactions in layered transition metal
dichalcogenides and serve as an intriguing example of how disorder within an
electronic crystal can give rise to pseudo-gap features
Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning
We show that denoising of 3D point clouds can be learned unsupervised,
directly from noisy 3D point cloud data only. This is achieved by extending
recent ideas from learning of unsupervised image denoisers to unstructured 3D
point clouds. Unsupervised image denoisers operate under the assumption that a
noisy pixel observation is a random realization of a distribution around a
clean pixel value, which allows appropriate learning on this distribution to
eventually converge to the correct value. Regrettably, this assumption is not
valid for unstructured points: 3D point clouds are subject to total noise, i.
e., deviations in all coordinates, with no reliable pixel grid. Thus, an
observation can be the realization of an entire manifold of clean 3D points,
which makes a na\"ive extension of unsupervised image denoisers to 3D point
clouds impractical. Overcoming this, we introduce a spatial prior term, that
steers converges to the unique closest out of the many possible modes on a
manifold. Our results demonstrate unsupervised denoising performance similar to
that of supervised learning with clean data when given enough training examples
- whereby we do not need any pairs of noisy and clean training data.Comment: Proceedings of ICCV 201
Single-image Tomography: 3D Volumes from 2D Cranial X-Rays
As many different 3D volumes could produce the same 2D x-ray image, inverting
this process is challenging. We show that recent deep learning-based
convolutional neural networks can solve this task. As the main challenge in
learning is the sheer amount of data created when extending the 2D image into a
3D volume, we suggest firstly to learn a coarse, fixed-resolution volume which
is then fused in a second step with the input x-ray into a high-resolution
volume. To train and validate our approach we introduce a new dataset that
comprises of close to half a million computer-simulated 2D x-ray images of 3D
volumes scanned from 175 mammalian species. Applications of our approach
include stereoscopic rendering of legacy x-ray images, re-rendering of x-rays
including changes of illumination, view pose or geometry. Our evaluation
includes comparison to previous tomography work, previous learning methods
using our data, a user study and application to a set of real x-rays
Joint Material and Illumination Estimation from Photo Sets in the Wild
Faithful manipulation of shape, material, and illumination in 2D Internet
images would greatly benefit from a reliable factorization of appearance into
material (i.e., diffuse and specular) and illumination (i.e., environment
maps). On the one hand, current methods that produce very high fidelity
results, typically require controlled settings, expensive devices, or
significant manual effort. To the other hand, methods that are automatic and
work on 'in the wild' Internet images, often extract only low-frequency
lighting or diffuse materials. In this work, we propose to make use of a set of
photographs in order to jointly estimate the non-diffuse materials and sharp
lighting in an uncontrolled setting. Our key observation is that seeing
multiple instances of the same material under different illumination (i.e.,
environment), and different materials under the same illumination provide
valuable constraints that can be exploited to yield a high-quality solution
(i.e., specular materials and environment illumination) for all the observed
materials and environments. Similar constraints also arise when observing
multiple materials in a single environment, or a single material across
multiple environments. The core of this approach is an optimization procedure
that uses two neural networks that are trained on synthetic images to predict
good gradients in parametric space given observation of reflected light. We
evaluate our method on a range of synthetic and real examples to generate
high-quality estimates, qualitatively compare our results against
state-of-the-art alternatives via a user study, and demonstrate
photo-consistent image manipulation that is otherwise very challenging to
achieve
Metappearance: Meta-Learning for Visual Appearance Reproduction
There currently are two main approaches to reproducing visual appearance
using Machine Learning (ML): The first is training models that generalize over
different instances of a problem, e.g., different images from a dataset. Such
models learn priors over the data corpus and use this knowledge to provide fast
inference with little input, often as a one-shot operation. However, this
generality comes at the cost of fidelity, as such methods often struggle to
achieve the final quality required. The second approach does not train a model
that generalizes across the data, but overfits to a single instance of a
problem, e.g., a flash image of a material. This produces detailed and
high-quality results, but requires time-consuming training and is, as mere
non-linear function fitting, unable to exploit previous experience. Techniques
such as fine-tuning or auto-decoders combine both approaches but are sequential
and rely on per-exemplar optimization. We suggest to combine both techniques
end-to-end using meta-learning: We over-fit onto a single problem instance in
an inner loop, while also learning how to do so efficiently in an outer-loop
that builds intuition over many optimization runs. We demonstrate this concept
to be versatile and efficient, applying it to RGB textures, Bi-directional
Reflectance Distribution Functions (BRDFs), or Spatially-varying BRDFs
(svBRDFs)
Plateau-reduced Differentiable Path Tracing
Current differentiable renderers provide light transport gradients with
respect to arbitrary scene parameters. However, the mere existence of these
gradients does not guarantee useful update steps in an optimization. Instead,
inverse rendering might not converge due to inherent plateaus, i.e., regions of
zero gradient, in the objective function. We propose to alleviate this by
convolving the high-dimensional rendering function that maps scene parameters
to images with an additional kernel that blurs the parameter space. We describe
two Monte Carlo estimators to compute plateau-free gradients efficiently, i.e.,
with low variance, and show that these translate into net-gains in optimization
error and runtime performance. Our approach is a straightforward extension to
both black-box and differentiable renderers and enables optimization of
problems with intricate light transport, such as caustics or global
illumination, that existing differentiable renderers do not converge on.Comment: Accepted to CVPR 2023. Project page and interactive demos at
https://mfischer-ucl.github.io/prdpt
3D Object Class Detection in the Wild
Object class detection has been a synonym for 2D bounding box localization
for the longest time, fueled by the success of powerful statistical learning
techniques, combined with robust image representations. Only recently, there
has been a growing interest in revisiting the promise of computer vision from
the early days: to precisely delineate the contents of a visual scene, object
by object, in 3D. In this paper, we draw from recent advances in object
detection and 2D-3D object lifting in order to design an object class detector
that is particularly tailored towards 3D object class detection. Our 3D object
class detection method consists of several stages gradually enriching the
object detection output with object viewpoint, keypoints and 3D shape
estimates. Following careful design, in each stage it constantly improves the
performance and achieves state-ofthe-art performance in simultaneous 2D
bounding box and viewpoint estimation on the challenging Pascal3D+ dataset
Deep Reflectance Maps
Undoing the image formation process and therefore decomposing appearance into
its intrinsic properties is a challenging task due to the under-constraint
nature of this inverse problem. While significant progress has been made on
inferring shape, materials and illumination from images only, progress in an
unconstrained setting is still limited. We propose a convolutional neural
architecture to estimate reflectance maps of specular materials in natural
lighting conditions. We achieve this in an end-to-end learning formulation that
directly predicts a reflectance map from the image itself. We show how to
improve estimates by facilitating additional supervision in an indirect scheme
that first predicts surface orientation and afterwards predicts the reflectance
map by a learning-based sparse data interpolation.
In order to analyze performance on this difficult task, we propose a new
challenge of Specular MAterials on SHapes with complex IllumiNation (SMASHINg)
using both synthetic and real images. Furthermore, we show the application of
our method to a range of image-based editing tasks on real images.Comment: project page: http://homes.esat.kuleuven.be/~krematas/DRM
Electronic self-organization in layered transition metal dichalcogenides
The interplay between different self-organized electronically ordered states and their relation to unconventional electronic properties like superconductivity constitutes one of the most exciting challenges of modern condensed matter physics. In the present thesis this issue is thoroughly investigated for the prototypical layered material 1T-TaS2 both experimentally and theoretically.
At first the static charge density wave order in 1T-TaS2 is investigated as a function of pressure and temperature by means of X-ray diffraction. These data indeed reveal that the superconductivity in this material coexists with an inhomogeneous charge density wave on a macroscopic scale in real space. This result is fundamentally different from a previously proposed separation of superconducting and insulating regions in real space. Furthermore, the X-ray diffraction data uncover the important role of interlayer correlations in 1T-TaS2.
Based on the detailed insights into the charge density wave structure obtained by the X-ray diffraction experiments, density functional theory models are deduced in order to describe the electronic structure of 1T-TaS2 in the second part of this thesis. As opposed to most previous studies, these calculations take the three-dimensional character of the charge density wave into account. Indeed the electronic structure calculations uncover complex orbital textures, which are interwoven with the charge density wave order and cause dramatic differences in the electronic structure depending on the alignment of the orbitals between neighboring layers. Furthermore, it is demonstrated that these orbital-mediated effects provide a route to drive semiconductor-to-metal transitions with technologically pertinent gaps and on ultrafast timescales.
These results are particularly relevant for the ongoing development of novel, miniaturized and ultrafast devices based on layered transition metal dichalcogenides. The discovery of orbital textures also helps to explain a number of long-standing puzzles concerning the electronic self-organization in 1T-TaS2 : the ultrafast response to optical excitations, the high sensitivity to pressure as well as a mysterious commensurate phase that is commonly thought to be a special phase a so-called “Mott phase” and that is not found in any other isostructural modification
- …